The present invention relates to a method and apparatus for recording of audio and video streams, and particularly, to a method and apparatus for synchronized recording of audio and video streams.
Methods and systems for synchronized recoding audio and video streams are known in the art. Generally, the encoding process consists of three stages. The first stage is digitizing the analog audio source and video source. The audio source is composed of a plurality of audio frames and each of the audio frames comprises a specific number of audio samples. Similarly, the video source is composed of a plurality of video frames. The second stage is compression of the digital audio and video frames to separately create an audio stream and a video stream. The audio stream is composed of the plurality of compressed audio frames. The video stream is composed of the plurality of compresses video frames. The third stage is multiplexing the audio and video streams into a single stream. The decoding process consists of inversing each of these stages and applying them in the reverse order.
Please refer to FIG. 1. FIG. 1 is a block diagram of a traditional encoding system 10. The encoding system 10 includes an audio encoder 20, a video encoder 30, and a multiplexer 40. The audio encoder 20 includes an audio analog to digital converter (A/D) 22 and an audio compressor 24. The video encoder 30 includes a video A/D 32 and a video compressor 34. The audio compressor 24 is connected to the audio A/D 22 and to the multiplexer 40. The video compressor 34 is connected to the video A/D 32 and to the multiplexer 40. An A/D converter is also known as a digitizer.
Unfortunately, when the audio signals output from the audio encoder 20 and the video signals output from the video encoder 30 reach the multiplexer 40, there is no way to know exactly when the audio and video signals were digitized. That is, audio signals that were digitized at a certain time should be associated with video signals that were digitized at the same time. Since the audio compressor 24 and the video compressor 34 do not always compress signals at the same rate, the compressed audio frame and its corresponding compressed video frame may not reach the multiplexer 40 at the same time. In addition, the timing information of digitalizing the analog audio source and video source in the audio A/D converter 22 and video A/D converter 32 are lost while the video and audio streams are inputted to the multiplexer 40, and therefore the video and audio streams will be out-of-synchronization.
To solve this problem, Sackstein, et al. proposes an encoding system in U.S. Pat. No. 6,744,815, which is herein incorporated by reference. Please refer to FIG. 2, which is a block diagram of an encoding system 100 according to the related art.
Encoding system 100 includes an audio encoder 110, a video encoder 130, an audio buffer 140, a video buffer 144, a controller 142, and a multiplexer 150.
The audio encoder 110 includes an audio A/D converter 114, an audio compressor 120, and an audio clock 112 connected to the audio A/D converter 114 and to an audio source. The video encoder 130 includes a video A/D converter 134, a video compressor 136, and a video clock 132 connected to the video A/D converter 134 and to a video source.
The audio buffer 140 is connected to the audio compressor 120, the controller 142, and the multiplexer 150. The video buffer 144 is connected to the video compressor 136, the controller 142, and the multiplexer 150. The audio compressor 120 includes an input buffer 122, for temporal storage of audio samples, and a CPU 124, connected therebetween. The controller 142 is further connected to the input buffer 122.
The audio A/D converter 114 provides digital audio samples to the compressor 120 at a rate dictated by its sampling audio clock 112. The compressor 120 has no knowledge of the pass of time except through the samples that arrive at its input.
As mentioned above, the audio compressor 24 and the video compressor 34 do not always compress signals at the same rate, and in order to obtain the encoding rate difference between the audio compressor 24 and the video compressor 34, the CPU 124 stores the compressed audio frames (audio samples) in the audio buffer 140 and the video compressor 136 stores the compresses video frames in the video buffer 144, and therefore, the controller 142 can know the encoding rate difference between the audio compressor 24 and the video compressor 34 by analyzing the number of audio samples in the audio buffer 140 and the number of the video frames in the video buffer 144 by increasing or decreasing the number of audio samples in the input buffer 122, the audio and video streams can be synchronized and multiplexed together in the multiplexer 150.
After analyzing the contents of the audio buffer 140 and the video buffer 144, the controller 142 provides feedback to the input buffer 122 for altering the audio samples contained in the input buffer 122. For instance, decrementing the write pointer in the input buffer 122 decreases the number of audio samples therein, while duplicating an audio sample in the input buffer 122 increases the number of audio samples therein.
Accordingly, the conventional encoding system 100 is only able to achieve synchronized recording of the audio and video steams by removing or adding audio samples stored in the input buffer 122 while the error of audio/video synchronization occurs.